I guess we need a comment here, at the very least.

But there may be a more general problem here. One would expect that after self.data = data.transform(domain), we would have self.data.domain is domain, while transform only ensures self.data.domain == domain. We should either change transform to (always) compare domains with is, not ==, or we can add a flag to specify how to compare domains.

@lanzagar, @astaric?

I would change transform to compare with is.

CMIIW, but changing Table.from_table to compare domains with is instead of == doesn't look like a trivial change. Also it by itself doesn't make OWColor work — the problem then arises in DomainConversion, which doesn't seem to know how to do the transformation. As a result, we get a data table with all values missing.

Before, as we checked with == if domains matched we called from_table_rows which just makes a selection of rows and keeps the provided domain. Now, if domain1 == domain2 but domain1 is not domain2 we need to make a domain conversion.

It seems like the core of the problem with all values missing is in how DomainConversion checks whether some variable is inside a domain. The relevant part of the code from domain.py:

    def __contains__(self, item):
        """
        Return `True` if the item (`str`, `int`, :class:`Variable`) is
        in the domain.
        """
        return item in self._indices

It seems like item in self._indices == False while item in list(self._indices.keys()) == True. Is this a result of how we compare variables?

Should we maybe just revert to self.data.domain = domain as it was before @janezd refactoring? If so, @janezd can you provide an explanation, since I'm not quite sure how variable's proxies work?

The first problem sounds like a broken hash on Variable.

As for the fix, I would rather move the data.domain = domain to transformmethod than have every caller do this explicitly.

@astaric but shouldn't then we fix Table.from_table as well to prevent from_table and transform to behave differently? Isn't data.transform currently just a syntactic sugar?

That is also possible. As we have changed all occurrences of Table.from_table with data.transform, I kind of consider it deprecated, but yeah, we can put the change there.

... and risk @janezd changing it back to data.transform the next time he sees this code? If data.transform is syntactic sugar it has to have the same behaviour as the thing that this sugar is coating.

... and risk @janezd changing it back to data.transform the next time he sees this code?

@janezd change to data.transform didn't break anything (transform and to_table behave the same way). Removing of the extra self.data.domain = domain is the change that broke things.

If data.transform is syntactic sugar it has to have the same behaviour as the thing that this sugar is coating.

Agree. Hence, if we are going to fix this in the core we need to do it in from_table. And while I'm all for changing to_table to compare domains with is instead of ==, I'm afraid this might cause other issues all over the place. As it has revealed some problems in domain's __contains__ which I'm not sure how to solve. Any help appreciated 🙈

And if anyone wishes to play around, here is the minimal example of the problem, after the change introduced in this PR:

import Orange
from Orange.data import Table


def _create_proxies(variables):
    part_vars = []
    for var in variables:
        if var.is_discrete or var.is_continuous:
            var = var.make_proxy()
            if var.is_discrete:
                var.values = var.values[:]
        part_vars.append(var)
    return part_vars


data = Table('iris')
domain = data.domain
domain = Orange.data.Domain(_create_proxies(domain.attributes),
                            _create_proxies(domain.class_vars),
                            _create_proxies(domain.metas))

new_data = data.transform(domain)
print(new_data)

The new_data contains all missing values since variables from the destination domain are not in the source domain and compute values are not set.